![All About BI !](/img/default-banner.jpg)
- 286
- 1 911 996
All About BI !
India
Приєднався 6 жов 2011
Software Engineer and a passionate teacher!
Learning does't need special brain. It just requires your special time!
Let's put some time off for learning everyday! Happy Learning.
Find the files that I use for DEMOs here:
github.com/CloudBIStack/DemoSourceFiles
Learning does't need special brain. It just requires your special time!
Let's put some time off for learning everyday! Happy Learning.
Find the files that I use for DEMOs here:
github.com/CloudBIStack/DemoSourceFiles
Azure Data Factory\\Synapse: Issues while Connecting to On-premise Server and fix for it
Older video on this - ua-cam.com/video/BsGa9iVtsRU/v-deo.htmlsi=8e5ojWSXikw0bHiD
ua-cam.com/video/ioVkkRS7lYo/v-deo.htmlsi=EVQhloyyhayxvl9x
ua-cam.com/video/ioVkkRS7lYo/v-deo.htmlsi=EVQhloyyhayxvl9x
Переглядів: 280
Відео
Azure Data Factory \\ Synapse Analytics - Delete folders older than 7 days!
Переглядів 4852 місяці тому
This video takes you through the steps we need to use for getting rid of folders in ADLS which are greater than 7 days
Find Latest Sub Folder in ADF\\Synapse
Переглядів 2823 місяці тому
Find Latest Sub Folder in ADF\\Synapse
Azure Data Factory\\Synapse Analytics - Deep dive into Schema Drift
Переглядів 5933 місяці тому
This video takes you through the Schema Drift and the possible usecases.
ADF\\ Synapse Analytics - Capture Metrics data from Data flow
Переглядів 3234 місяці тому
In real time, we will have the need to capture certain details from data flow execution, like rows read, rows written, status of data flow, etc. This video takes you through the output json and explains the parts of the output that we need to use for metrics. Reference: learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-data-flow-activity
ADF\\Synapse: Relialize the power of Wildcard based file copy!
Переглядів 3365 місяців тому
ADF\\Synapse: Relialize the power of Wildcard based file copy!
ADF\\Synapse: Replace space in column names just like that, using Rule-based mapping
Переглядів 3975 місяців тому
In Real time, we get to receive source data where column names might have spaces in it. In this video we are discussing how cool it is to replace spaces with in column names with a simple expression!
ADF\\Synapse: Interpret Script activity output json and write expressions to access column values
Переглядів 2735 місяців тому
This video takes you through the steps to interpret output of the script activity and create dynamic expressions to access individual elements of resultsets.
ADF\\Synapse: Pass Pipleline Param to DataFlow Param and use it in Source query 2 filter Src data
Переглядів 2905 місяців тому
This video takes you through the steps required to pass value to a data flow parameter from Pipeline Parameter. And in turn, using the data flow parameter in the source query, to filter records in Source SQL table.
ADF\\Synapse - Use REST API to know pipeline run status
Переглядів 1,3 тис.6 місяців тому
This video takes you through the steps \ details required to use a web activity to get pipeline run details. Documentation URL: learn.microsoft.com/en-us/rest/api/datafactory/pipeline-runs/get?view=rest-datafactory-2018-06-01
ADF\\Synapse - Copy files incrementally, based on Last modified date
Переглядів 5816 місяців тому
This video explains the files from one container to the another, based on last modified timestamp stored in a sql table until current time.
Azure Data Factory\\Synapse:Load from different XL files with different names to Az. SQL tables
Переглядів 3467 місяців тому
This video takes you through the activities we need to use to load different XL files, each with different sheet name to different SQL table dynamically. Get excel sheet names dynamically: ua-cam.com/video/giJhP6LiOgI/v-deo.html ua-cam.com/video/wmPzwrJIHc4/v-deo.html
Pricing\\ Costing Tips & Understanding - Detailed information on where you can cut down costs
Переглядів 5177 місяців тому
Pricing\\ Costing Tips & Understanding - Detailed information on where you can cut down costs
ADF\\Synapse: Store metadata of file in Az.Storage account
Переглядів 4558 місяців тому
This video takes you through the steps we need to follow in order to capture the meta data of file and store it in another file in blob\data lake.
ADF\\Synapse: Validate Email ID using REGEX in data flow
Переглядів 4988 місяців тому
ADF\\Synapse: Validate Email ID using REGEX in data flow
ADF\\Synapse : Ignore XML with no records or Process XML that has records
Переглядів 4939 місяців тому
ADF\\Synapse : Ignore XML with no records or Process XML that has records
Azure Data Factory\\Synapse: Decide name of additional column and its value dynamically
Переглядів 5639 місяців тому
Azure Data Factory\\Synapse: Decide name of additional column and its value dynamically
Scenario Based Question: Trigger exactly at 6am-8 am with two triggers
Переглядів 65310 місяців тому
Scenario Based Question: Trigger exactly at 6am-8 am with two triggers
Azure Data Factory - Take first row\\column from file
Переглядів 1,2 тис.10 місяців тому
Azure Data Factory - Take first row\\column from file
Azure Data Factory - Iterate through files in folder and get file structure
Переглядів 3,7 тис.11 місяців тому
Azure Data Factory - Iterate through files in folder and get file structure
ADF\\Synapse Analytics - Pass dynamic value to filename in Sink in Copy Activity & Data flow
Переглядів 3,8 тис.11 місяців тому
ADF\\Synapse Analytics - Pass dynamic value to filename in Sink in Copy Activity & Data flow
Azure Data Factory : Use Web Activity to Call child pipeline & Pass Parent Run id via Body
Переглядів 1,3 тис.11 місяців тому
Azure Data Factory : Use Web Activity to Call child pipeline & Pass Parent Run id via Body
Azure Functions : How to convert .csv to .xlsx using Az Funtion & Pandas
Переглядів 2,5 тис.Рік тому
Azure Functions : How to convert .csv to .xlsx using Az Funtion & Pandas
Azure Data Factory\\Synapse: Compute sum of integer elements in array
Переглядів 931Рік тому
Azure Data Factory\\Synapse: Compute sum of integer elements in array
Azure Functions: Use Azure Function & get XL Sheet names
Переглядів 649Рік тому
Azure Functions: Use Azure Function & get XL Sheet names
Azure Data Factory - Create Data factory pipeline using Python
Переглядів 3,8 тис.Рік тому
Azure Data Factory - Create Data factory pipeline using Python
Azure Data Factory\\Synapse Analytics : Fail the pipeline if no files matching wild card pattern
Переглядів 1,3 тис.Рік тому
Azure Data Factory\\Synapse Analytics : Fail the pipeline if no files matching wild card pattern
ADF\\ Synapse: Can we type cast value of Addition Column in Copy Activity?
Переглядів 497Рік тому
ADF\\ Synapse: Can we type cast value of Addition Column in Copy Activity?
Ma'am how to do transfermation of folders to folders. in source i have many folders under that i have files so those folders need to transfer target dynamiclly
hello mam, Mostly we write data into cosmos, it will has lot of nested column structure so how will you address his.
This video is super useful for beginners. Thanks a lot❤
Startdate and enddate as yyyyMM format give some example pls
please show how to install python packages on azure batch account for executing python script in data faotory please
Please show how to install packages on the batch account through task. No one covered this.
I thank you so much for the video. It saved my day. I am really thankful 🙏
What about pip install? Can you do that ma’am?
What if our sink destination is an SQL table?? How can we create a dynamic table name using the original file name of the data source (for an excel data source)
hello, i want to load the multiple tables from local database"mysql workbench" to azure lake gen 2 at a time can you tell me how can we do it? if possible make a video on it
Can you do this for Python please?
We can supply the array variable to a stored proc and use sql computation to sort and provide the latest file name.
Yes. Right. But some projects don't want to invest on azure SQL. They can leverage this.
Thank you for this video
one of the best explanations I have come across in UA-cam. Thank you
Glad you feel so ☺️☺️
Thank you so much for this video!!!! 🙏 I needed to iterate through a directory on a network drive and this is the information I needed.
Glad it helped.
Good explanation
Could you please.. tell how to use these steps in Azure synapse analytics. Because while I use the same url in web activity it through error..
Good Resolution is not good
Best adf tutorial !
Glad it is of help 😊
Mam , Python is already installed on the VM or we need to write the code in start task . Or write any other script to install python first
Code?
code?
Useful information 🎉
Great example of removing files, will use it now!
❤❤❤
Thanks For brining this up to the attention.
From where can i get dataset
Asking for Username and password when creating linked service. My personal laptop Username and password not working
Pls Enablelocalaccess from powershell cmd
can we keep on-prem details in azure key vault , if yes then how?
Super clear and reasonable scenarios
Is there any new training session started on Azure Devops Madam
Mam I want to get all the column names and fetch those column values from an API , how can I do it?
Is there in built option is there copy activity
How can we return this OutputtoADF value to a calling notebook? If suppose I am running a for loop in a diff notebook to call the processing notebook how will the error return back to it?
Hi, Thanks for posting this video. Can you please clarify how you ensured the files were split by country?
After first hit of the execution its need to wait 5 or 10 mins then need to execute the second one
Hi I have some questions here Example i have 10 items to pass from lookup to forloop activitie but this 10 times need to run parallel but if first one run after 10 mins need to run second item how we can set the time difference for each item execution
Video was so helpful and understandable. Good job.
Thanks for Sharing
can we try this on free tier account ?
Yes boss
@@AllAboutBI thank you
Very nice explanation , keep making good videos like this , appreciated
Hi Madam, Your providing any Azure data Engineering training. I need 1:1 training. Please suggest
Nopes, not at the moment bro
Thank you so much for this, exactly one of my tasks in project.
hey wonderful explanation whats your email
Thank you for the feedback. funlearn0007@gmail.com
thank you so much, very helpful video 👌
Thanks 🙏
I want to load XML file into MS Fabric ? Any idea?
How can I export schema of tables from SQL Server database to sink in csv file format and store in azure blob storage using ADF synapse?
How can you use replace double quotes on all the columns of all thr tables as part of cleansing using adf