All About BI !
All About BI !
  • 286
  • 1 911 996

Відео

Azure Data Factory \\ Synapse Analytics - Delete folders older than 7 days!
Переглядів 4852 місяці тому
This video takes you through the steps we need to use for getting rid of folders in ADLS which are greater than 7 days
Find Latest Sub Folder in ADF\\Synapse
Переглядів 2823 місяці тому
Find Latest Sub Folder in ADF\\Synapse
Azure Data Factory\\Synapse Analytics - Deep dive into Schema Drift
Переглядів 5933 місяці тому
This video takes you through the Schema Drift and the possible usecases.
ADF\\ Synapse Analytics - Capture Metrics data from Data flow
Переглядів 3234 місяці тому
In real time, we will have the need to capture certain details from data flow execution, like rows read, rows written, status of data flow, etc. This video takes you through the output json and explains the parts of the output that we need to use for metrics. Reference: learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-data-flow-activity
ADF\\Synapse: Relialize the power of Wildcard based file copy!
Переглядів 3365 місяців тому
ADF\\Synapse: Relialize the power of Wildcard based file copy!
ADF\\Synapse: Replace space in column names just like that, using Rule-based mapping
Переглядів 3975 місяців тому
In Real time, we get to receive source data where column names might have spaces in it. In this video we are discussing how cool it is to replace spaces with in column names with a simple expression!
ADF\\Synapse: Interpret Script activity output json and write expressions to access column values
Переглядів 2735 місяців тому
This video takes you through the steps to interpret output of the script activity and create dynamic expressions to access individual elements of resultsets.
ADF\\Synapse: Pass Pipleline Param to DataFlow Param and use it in Source query 2 filter Src data
Переглядів 2905 місяців тому
This video takes you through the steps required to pass value to a data flow parameter from Pipeline Parameter. And in turn, using the data flow parameter in the source query, to filter records in Source SQL table.
ADF\\Synapse - Use REST API to know pipeline run status
Переглядів 1,3 тис.6 місяців тому
This video takes you through the steps \ details required to use a web activity to get pipeline run details. Documentation URL: learn.microsoft.com/en-us/rest/api/datafactory/pipeline-runs/get?view=rest-datafactory-2018-06-01
ADF\\Synapse - Copy files incrementally, based on Last modified date
Переглядів 5816 місяців тому
This video explains the files from one container to the another, based on last modified timestamp stored in a sql table until current time.
Azure Data Factory\\Synapse:Load from different XL files with different names to Az. SQL tables
Переглядів 3467 місяців тому
This video takes you through the activities we need to use to load different XL files, each with different sheet name to different SQL table dynamically. Get excel sheet names dynamically: ua-cam.com/video/giJhP6LiOgI/v-deo.html ua-cam.com/video/wmPzwrJIHc4/v-deo.html
Pricing\\ Costing Tips & Understanding - Detailed information on where you can cut down costs
Переглядів 5177 місяців тому
Pricing\\ Costing Tips & Understanding - Detailed information on where you can cut down costs
ADF\\Synapse: Store metadata of file in Az.Storage account
Переглядів 4558 місяців тому
This video takes you through the steps we need to follow in order to capture the meta data of file and store it in another file in blob\data lake.
ADF\\Synapse: Validate Email ID using REGEX in data flow
Переглядів 4988 місяців тому
ADF\\Synapse: Validate Email ID using REGEX in data flow
ADF\\Synapse : Ignore XML with no records or Process XML that has records
Переглядів 4939 місяців тому
ADF\\Synapse : Ignore XML with no records or Process XML that has records
Azure Data Factory\\Synapse: Decide name of additional column and its value dynamically
Переглядів 5639 місяців тому
Azure Data Factory\\Synapse: Decide name of additional column and its value dynamically
Scenario Based Question: Trigger exactly at 6am-8 am with two triggers
Переглядів 65310 місяців тому
Scenario Based Question: Trigger exactly at 6am-8 am with two triggers
Azure Data Factory - Take first row\\column from file
Переглядів 1,2 тис.10 місяців тому
Azure Data Factory - Take first row\\column from file
Azure Data Factory - Iterate through files in folder and get file structure
Переглядів 3,7 тис.11 місяців тому
Azure Data Factory - Iterate through files in folder and get file structure
ADF\\Synapse Analytics - Pass dynamic value to filename in Sink in Copy Activity & Data flow
Переглядів 3,8 тис.11 місяців тому
ADF\\Synapse Analytics - Pass dynamic value to filename in Sink in Copy Activity & Data flow
Azure Data Factory : Use Web Activity to Call child pipeline & Pass Parent Run id via Body
Переглядів 1,3 тис.11 місяців тому
Azure Data Factory : Use Web Activity to Call child pipeline & Pass Parent Run id via Body
Azure Functions - Basics Part II
Переглядів 77611 місяців тому
Azure Functions - Basics Part II
Azure Functions - Basics - Part I
Переглядів 3,9 тис.Рік тому
Azure Functions - Basics - Part I
Azure Functions : How to convert .csv to .xlsx using Az Funtion & Pandas
Переглядів 2,5 тис.Рік тому
Azure Functions : How to convert .csv to .xlsx using Az Funtion & Pandas
Azure Data Factory\\Synapse: Compute sum of integer elements in array
Переглядів 931Рік тому
Azure Data Factory\\Synapse: Compute sum of integer elements in array
Azure Functions: Use Azure Function & get XL Sheet names
Переглядів 649Рік тому
Azure Functions: Use Azure Function & get XL Sheet names
Azure Data Factory - Create Data factory pipeline using Python
Переглядів 3,8 тис.Рік тому
Azure Data Factory - Create Data factory pipeline using Python
Azure Data Factory\\Synapse Analytics : Fail the pipeline if no files matching wild card pattern
Переглядів 1,3 тис.Рік тому
Azure Data Factory\\Synapse Analytics : Fail the pipeline if no files matching wild card pattern
ADF\\ Synapse: Can we type cast value of Addition Column in Copy Activity?
Переглядів 497Рік тому
ADF\\ Synapse: Can we type cast value of Addition Column in Copy Activity?

КОМЕНТАРІ

  • @arjunshetty7974
    @arjunshetty7974 День тому

    Ma'am how to do transfermation of folders to folders. in source i have many folders under that i have files so those folders need to transfer target dynamiclly

  • @RaviRaj-ok3fu
    @RaviRaj-ok3fu 3 дні тому

    hello mam, Mostly we write data into cosmos, it will has lot of nested column structure so how will you address his.

  • @sam1991jit
    @sam1991jit 6 днів тому

    This video is super useful for beginners. Thanks a lot❤

  • @Khader_views
    @Khader_views 8 днів тому

    Startdate and enddate as yyyyMM format give some example pls

  • @DiacoNori
    @DiacoNori 8 днів тому

    please show how to install python packages on azure batch account for executing python script in data faotory please

  • @DiacoNori
    @DiacoNori 8 днів тому

    Please show how to install packages on the batch account through task. No one covered this.

  • @vamsikrish3537
    @vamsikrish3537 8 днів тому

    I thank you so much for the video. It saved my day. I am really thankful 🙏

  • @DiacoNori
    @DiacoNori 9 днів тому

    What about pip install? Can you do that ma’am?

  • @louisar4227
    @louisar4227 10 днів тому

    What if our sink destination is an SQL table?? How can we create a dynamic table name using the original file name of the data source (for an excel data source)

  • @vamshikrishna1500
    @vamshikrishna1500 14 днів тому

    hello, i want to load the multiple tables from local database"mysql workbench" to azure lake gen 2 at a time can you tell me how can we do it? if possible make a video on it

  • @mohitupadhayay1439
    @mohitupadhayay1439 18 днів тому

    Can you do this for Python please?

  • @PrudhviT-wq5ze
    @PrudhviT-wq5ze 19 днів тому

    We can supply the array variable to a stored proc and use sql computation to sort and provide the latest file name.

    • @AllAboutBI
      @AllAboutBI 17 днів тому

      Yes. Right. But some projects don't want to invest on azure SQL. They can leverage this.

  • @shashankagrahari
    @shashankagrahari 20 днів тому

    Thank you for this video

  • @vedanthasm2659
    @vedanthasm2659 21 день тому

    one of the best explanations I have come across in UA-cam. Thank you

    • @AllAboutBI
      @AllAboutBI 21 день тому

      Glad you feel so ☺️☺️

  • @user-lx7qf6rj5f
    @user-lx7qf6rj5f 22 дні тому

    Thank you so much for this video!!!! 🙏 I needed to iterate through a directory on a network drive and this is the information I needed.

  • @venumanam8826
    @venumanam8826 22 дні тому

    Good explanation

  • @vishnupriya7196
    @vishnupriya7196 22 дні тому

    Could you please.. tell how to use these steps in Azure synapse analytics. Because while I use the same url in web activity it through error..

  • @mdaslamknl
    @mdaslamknl 25 днів тому

    Good Resolution is not good

  • @reshmasanghvi3805
    @reshmasanghvi3805 27 днів тому

    Best adf tutorial !

    • @AllAboutBI
      @AllAboutBI 27 днів тому

      Glad it is of help 😊

  • @revatinebhnani8846
    @revatinebhnani8846 28 днів тому

    Mam , Python is already installed on the VM or we need to write the code in start task . Or write any other script to install python first

  • @nitinlad6669
    @nitinlad6669 29 днів тому

    Code?

  • @nitinlad6669
    @nitinlad6669 29 днів тому

    code?

  • @user-hd9lk6qx4u
    @user-hd9lk6qx4u 29 днів тому

    Useful information 🎉

  •  Місяць тому

    Great example of removing files, will use it now!

  • @crazylal6886
    @crazylal6886 Місяць тому

    ❤❤❤

  • @gowthamvishva6316
    @gowthamvishva6316 Місяць тому

    Thanks For brining this up to the attention.

  • @shriyankarath5388
    @shriyankarath5388 Місяць тому

    From where can i get dataset

  • @ArvindKumar-he4on
    @ArvindKumar-he4on Місяць тому

    Asking for Username and password when creating linked service. My personal laptop Username and password not working

    • @AllAboutBI
      @AllAboutBI Місяць тому

      Pls Enablelocalaccess from powershell cmd

  • @user-js5rg4nx5r
    @user-js5rg4nx5r Місяць тому

    can we keep on-prem details in azure key vault , if yes then how?

  • @ranganathmaimal
    @ranganathmaimal Місяць тому

    Super clear and reasonable scenarios

  • @RajeshKumar-bk7wy
    @RajeshKumar-bk7wy Місяць тому

    Is there any new training session started on Azure Devops Madam

  • @user-xw5le5yd3t
    @user-xw5le5yd3t Місяць тому

    Mam I want to get all the column names and fetch those column values from an API , how can I do it?

  • @Anushka-qz6dn
    @Anushka-qz6dn Місяць тому

    Is there in built option is there copy activity

  • @mohitupadhayay1439
    @mohitupadhayay1439 Місяць тому

    How can we return this OutputtoADF value to a calling notebook? If suppose I am running a for loop in a diff notebook to call the processing notebook how will the error return back to it?

  • @uttapa22
    @uttapa22 Місяць тому

    Hi, Thanks for posting this video. Can you please clarify how you ensured the files were split by country?

  • @bmreddy316
    @bmreddy316 Місяць тому

    After first hit of the execution its need to wait 5 or 10 mins then need to execute the second one

  • @bmreddy316
    @bmreddy316 Місяць тому

    Hi I have some questions here Example i have 10 items to pass from lookup to forloop activitie but this 10 times need to run parallel but if first one run after 10 mins need to run second item how we can set the time difference for each item execution

  • @vigneshramanathan3088
    @vigneshramanathan3088 Місяць тому

    Video was so helpful and understandable. Good job.

  • @vijaybodkhe8379
    @vijaybodkhe8379 Місяць тому

    Thanks for Sharing

  • @aadhilimam8253
    @aadhilimam8253 Місяць тому

    can we try this on free tier account ?

  • @anilchoubey7155
    @anilchoubey7155 Місяць тому

    Very nice explanation , keep making good videos like this , appreciated

  • @srikantha2884
    @srikantha2884 Місяць тому

    Hi Madam, Your providing any Azure data Engineering training. I need 1:1 training. Please suggest

    • @AllAboutBI
      @AllAboutBI Місяць тому

      Nopes, not at the moment bro

  • @lakshmiprasanna2373
    @lakshmiprasanna2373 Місяць тому

    Thank you so much for this, exactly one of my tasks in project.

  • @shyamsundarmolli
    @shyamsundarmolli Місяць тому

    hey wonderful explanation whats your email

    • @AllAboutBI
      @AllAboutBI Місяць тому

      Thank you for the feedback. funlearn0007@gmail.com

  • @egdteg7981
    @egdteg7981 Місяць тому

    thank you so much, very helpful video 👌

  • @itversityitversity7690
    @itversityitversity7690 Місяць тому

    I want to load XML file into MS Fabric ? Any idea?

  • @nareshshah2094
    @nareshshah2094 Місяць тому

    How can I export schema of tables from SQL Server database to sink in csv file format and store in azure blob storage using ADF synapse?

  • @Ptelearn4free
    @Ptelearn4free Місяць тому

    How can you use replace double quotes on all the columns of all thr tables as part of cleansing using adf